Engineering posts about Service Discovery
Curated summaries and key learnings for engineers working with Service Discovery.
How AI-Powered Attacks Led Salesforce to Reinvent Hyperscale DDoS Defense
The article outlines the development of DREAM, an AI-powered DDoS mitigation platform designed by Salesforce to enhance its defense against hyperscale DDoS attacks. It highlights the architectural...
Databricks Network Configuration delivery to Tens of Millions of Serverless VMs
The article discusses the re-architecture of Databricks' network configuration delivery system for its serverless platform, which launches tens of millions of virtual machines (VMs) daily. The...
How and Why Netflix Built a Real-Time Distributed Graph: Part 3 — Querying the graph with gRPC…
In this article, Netflix engineers Nilesh Mishra and Ajit Koti detail the architecture and design choices behind their Real-Time Distributed Graph (RDG), particularly focusing on the querying layer...
Building Service Topology at Scale: Architecture, Challenges, and Lessons Learned
This article delves into the complexities of constructing a real-time service topology at Netflix scale, focusing on the architectural decisions that facilitate efficient data processing and system...
From Silos to Service Topology: Why Netflix Built a Real-Time Service Map
The article outlines Netflix's development of a real-time service topology map to improve observability and troubleshooting in its microservices architecture. It highlights the challenges faced by...
How Netflix Simplified Batch Compute with Kueue
The article outlines Netflix's transition from its custom Compute Managed Batch (CMB) solution to Kueue, a cloud-native job queueing system designed for batch workloads. This migration was motivated...
Sitar-agent: Building a reliable dynamic configuration sidecar at scale
The article discusses the development of Sitar-agent, a Kubernetes sidecar designed to ensure reliable dynamic configuration delivery at scale for Airbnb's services. It outlines the configuration...
Lights Out, Systems On: Validating Instant Power Loss Readiness
The article introduces the Instantaneous PowerLoss Storm, a testing paradigm developed by Meta to prepare data centers for zero-notice power loss scenarios. It outlines the strategies implemented to...
Rethinking Distributed Systems for Serverless Performance and Reliability
The article explores the evolution of serverless compute for Apache Spark, addressing long-standing architectural challenges that have hindered performance and reliability. It emphasizes the need for...
From SSH to REST: A Security-Driven Modernization of Slack’s EMR Data Pipelines
The article outlines Slack's transition from a legacy SSH-based architecture to a modern REST-based job submission system for its data pipelines. Initially, the reliance on SSH created significant...
Rearchitecting the Workflows control plane for the agentic era
The article discusses the rearchitecting of the Workflows control plane to accommodate a shift towards agent-triggered workflows, necessitated by the increasing demand for durable execution engines...
Building a Distributed Persistent Queue That Scaled AI Workloads 5x Under LLM Rate Limits
The article discusses the engineering of a distributed persistent queue that orchestrates AI workloads and human workflows within strict infrastructure limits. It highlights the challenges of scaling...